Inventing Indicators of Interdisciplinarity

Partners:

Centre for Interdisciplinary Methodologies (University of Warwick)
Centre for Science and Technology Studies (University of Leiden)

2018 - …

Participants: Noortje Marres, Sarah de Rijcke, Ismael Rafols, Celia Lury, Ludo Waltman, Rodrigues Costa Comesana, Sybille Lammes, Nerea Calvillo, Tjitske Holtrop, Anne Beaulieu, Greg McInerney, Thomas Franssen.

Inventing Indicators of Interdisciplinarity is an experimental pilot project that is currently under development as a collaboration between the Centre for Science & Technology Studies (CWTS) at the University Leiden and the Centre for Interdisciplinary Methodologies (CIM) at the University of Warwick. The aim is to make links between scientometrics, visualisation, evaluative inquiry and playful methods (design research) in order to devise alternative, more relational approaches to interdisciplinarity and its e-valuation.

The project starts from the insight that interdisciplinary research is often not evaluated as positively as disciplinary research both in peer review panels for journals and grants, evaluation exercises such as the REF, and in wider research cultures. The increasing use of performance indicators such as the Journal Impact Factor as part of evaluation processes often also have negative consequences for interdisciplinary research. On a more general level, performance indicators, such as those derived from citation data, have in recent years been criticized as driving the metricization of research culture (De Rijcke et al, 2016). As Ismael Rafols put it (2018: “[A] compelling argument of ‘what went wrong with indicators’ (Barré, 2018) [..] is that, although indicators were developed in the spirit of enlightenment as tools that would inform decision-making, they have become ‘ignorance producing devices’, in the sense that they are used as horse blinkers that reduce the issues taken into consideration”. However, indicators have at the same time been identified as a key site of methodological innovation towards enabling more generative, inclusive and interactive ways of developing research agenda's, evaluation practices and fostering research culture (Holtrop, 2018).

This project therefore takes up the following challenge: Can we devise and deploy indicators in ways that do not bracket but enable the exploration of "the differences and asymmetries between disciplines, in relation to specific problems, in specific places, with specific materials?” (Lury, Lammes, Uprichard et al, 2018). Our starting point, in doing so, is that indicators are not only instruments of measurement, they also present potentially powerful devices of engagement (Rafols, 2018; Gerlitz and Lury, 2016), capable of inviting experts, interested parties and audiences into a problem definition or relevance proposition. It also follows from this that indicators are designed entities: The process of visualisation and interpretation of an indicator is structured – or not - so as to facilitate some interpretations rather than others, assemble some communities rather than others. Rather than working through these issues in the abstract, however, in this project we develop scripts for indicating interdisciplinarity focusing on specific areas of research and innovation, such as artificial intelligence (AI).

In devising indicators of interdisciplinarity for this area, we will review and explore the potential of scientometric, cartographic and design methods to indicate interdisciplinarity in AI research and innovation in creative and engaging ways that support and highlight interdisciplinary research rather than suppress it.

Recent paper : Marres, N. & de Rijcke, S. (2020). From indicators to indicating interdisciplinarity: A participatory mapping methodology for research communities in-the-making. Quantitative Science Studies, first published online 29 June 2020, 1041-1055.